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Abstract. In this paper, we propose the development of a web-based, intelligent instruction system 
to help elementary school students for mathematical computation. We concentrate on the intelli¬ 
gence facilities which support diagnosis and advice. The existing web-based instruction systems 
merely give information on whether the learners’ replies are ‘correct’ or ‘incorrect’, and only offer 
evaluations of the learners’ results in terms of points. What is needed is a web-based instruction 
system that diagnoses the learner’s comprehension status, and provides cause: why did the learner 
make the error? Our system has a facility that analyses the learner’s weak points and has the ability 
to diagnose the cause of the error, giving advice to the learners and more detailed error informa¬ 
tion than extant systems. By accumulating user behavior and analyzing the learner’s responses, our 
system provides relevant, individualized information, along with advice for the learners. 

Keywords: diagnosis facilities, intelligent CAI, web-based courseware, e-leaming. 


1. Introduction 

With the rapid growth of computer and Internet technologies, e-learning has become 
a major trend in the computer-assisted teaching and learning fields (Chen et al., 2007). 
The web has the advantages of flexibility, free browsing, and easy use; and thus offers 
a very powerful medium for data sharing. These advantages suggest the possibility of 
a highly efficient instruction tool. Distance learning is in increasing demand, so web- 
based instruction systems have not ceased to expand in popularity. 

There are several advantages to the web-based approach on the Internet compared 
with traditional classroom teaching (Huang et al., 2007; Khan, 2001): 

• The computer networks offer the learners learning opportunities free from the lim¬ 
itations of time and space. 

• Learning processes can be operated and monitored through computerized systems; 
teaching management can thus proceed more efficiently. 
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• Auxiliary learning tools are offered to speed up the delivery of instructional infor¬ 
mation and to assist more effective interaction among the members. 

• Collaborative teaching and cooperative learning can be done by teachers and learn¬ 
ers respectively by constructing communities on the website as learning platforms. 

• Learning progress and achievements for each learner are disclosed to the teacher 
and all team members. This will allow them to offer guidance to the laggard learn¬ 
ers and help the teacher to analyze the learners’ behavior. 

• The accumulated achievements offer not only an excellent opportunity for the 
learners to inspect and learn from each other, but also extend the knowledge range 
more deeply and widely. 

• The information can be kept up to date. It is difficult to obtain educational material 
that is less than three months old using other media; but, using web software, we 
access educational information that was built yesterday. 

• There is a vast amount of information available. We can utilize this large amount 
of information for instruction on the Internet. 

Many web-based, instruction systems have already been developed, but the result has 
been disappointing; many are not widely used, and there are far-reaching limitations and 
problems. Here are some of them (Khan, 2001): 

• They do not fully utilize computer network advantages although the web provides 
an opportunity to develop new learning experiences not previously possible for 
learners. 

• They are dependent on a strong hypertext model, so they have many branches. 
Learners just choose a link and get content. The extant systems do not consider the 
learner’s status of comprehension or level. Ideally, a system needs to check learner 
understanding status frequently and should generate content dynamically. 

• In existing web-based instruction systems, when learners make errors they simply 
receive an error message, but do not necessarily know the cause of the error. Sys¬ 
tems need to diagnose why learners make errors, and they need to give appropriate 
corrective knowledge to these learners. 

• They have difficulty in ensuring interaction between the instructional system and 
learner, so they do not guarantee learner activity. We need to improve interaction 
in web-based educational software. 

Thus, the web remains problem ridden in respect to applications in the field of edu¬ 
cation. Although many web-based learning techniques have been proposed to assist web- 
based learning, little research has taken into account the issue of learning diagnosis in 
the development of web-based instruction systems. It is significant to diagnose learners’ 
understanding in e-learning and courseware. This guarantees quality and sustainability 
of learning, but it is particularly difficult in self-learning because learners must judge 
their own understanding levels by themselves. Hence, systems with a diagnosis function 
such as Intelligent Tutoring Systems (Gunel, 2010) are required. An artificial intelligence 
model is effective for such a problem (Namba, 2012). 

Learning diagnosis aims to identify learner’s misconceptions in order to improve 
learning performance during learning processes. Generally, by discovering the learner’s 
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misconceptions, important feedback can be provided by web-based instruction systems 
that can perform remedial learning or revise instruction strategies (Kim and Lee, 1999). 

In our system, we have attempted to minimize the above problems. We propose a web- 
based instruction system in mathematics for elementary school students that offer flexibil¬ 
ity to the learners with respect to time and place of access. Such a system would analyze 
learner shortcomings and be able to diagnose error cause, offering relevant advice and 
information for the avoidance and correction of future errors. The system would pro¬ 
vide the learner with more detailed information than is available with current systems. In 
addition, our system would store records of user behavior in a database and offer a com¬ 
parison between learners, giving the learner a sense of his or her position relative to other 
learners using the system. But our system has a limitation in that it can only be applied to 
computation in mathematics. 

The remainder of the paper is organized as follows: Section 2 gives background theory 
and related works. Section 3 gives a description of the architecture of the proposed web- 
based, intelligent instruction system. We will show the details of the intelligent diagnosis 
mechanism with examples. Conclusions and proposals for future work can be found in 
Section 4. 


2. Related Works 

We reviewed instructional design, existed Intelligent system and diagnosis system. 

First of all, this study paid attention to Instructional Design (ID). Because Instruc¬ 
tional Design is part of Instructional Science, which encompasses theories, models, 
methodologies, and tools for instruction (Mizoguchi and Bourdeau, 2000). ID is “an en¬ 
gineering activity for which the artefact is some instructional product conceived to help 
a learner acquire some knowledge or skill” (Merrill, 2001). This activity applies strate¬ 
gies and techniques derived from behavioural, cognitive, and constructivist theories to the 
solution of instructional problems (Mizoguchi and Bourdeau, 2000). 

Instructional Design theories are prescriptions for designing instructional products to 
optimize the learning outcome (Merrill, 2001; Mukhopadhyay and Madhu, 2001). These 
theories describe methods of instruction together with situations in which those methods 
should be used (Reigeluth, 1999). From a pragmatic viewpoint, the underlying questions 
about instructional design are what to teach and how to teach (Dick etai, 2000). Research 
in ID has constantly focused on the dualism between teaching theory and practice in order 
to develop effective, efficient and appealing products for instruction. 

Computing science has investigated ways of achieving this goal using software tools. 
Implemented computer-based tools range from intelligent tutoring systems to web based 
e-learning platforms (Gaudioso and Boticario, 2003). A common feature of all these sys¬ 
tems is that they either implicitly or explicitly use knowledge about Instructional De¬ 
sign. However, incorporating knowledge in a computer program is a complex and labour- 
intensive task: firstly, ID knowledge must be acquired; secondly, procedures to use this 
knowledge must be implemented. Rather than building new systems from scratch, the 
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ID knowledge should be represented in such a way that it can be reused by different 
applications (Mizoguchi and Bourdeau, 2000). 

Especially, Meisel and Compatangelo (2004) describes an architecture for the usage 
of ID knowledge in intelligent instructional systems. In contrast with other architectures, 
ontologies are used to represent ID knowledge about both what to teach and how to 
teach. Moreover, set-theoretic reasoning is used for the provision of inferential services. 
In particular, the paper shows how set-theoretic deductions can be applied to support the 
modelling of ID knowledge bases, to retrieve suitable teaching methods from them, and 
to detect errors in a training design. 

Second, we gave attention to existed intelligent instruction system based on web 
(Chen et al., 2005; Cheung et al., 2003; Depradine, 2003; Huang et al., 2007; Zhang 
et al., 2001). 

A summary of requirements for intelligent systems was presented by Hartley and 
Sleeman in the early 1970s. They argued that these systems must possess: knowledge of 
the learner (student model), knowledge of the domain (expert model), and knowledge of 
teaching strategies (pedagogical model). It is interesting to note that this simple list has 
not changed in more than three decades; however, advances have been made in each of 
the three areas. All of the computer-resident knowledge marks a radical shift from earlier 
knowledge-free, computed-assisted instructional programs. Furthermore, the ability to 
diagnose students’ errors and adapt instruction based on the diagnosis represents a key 
difference between intelligent versus other computer-based systems, such as simulations. 
Intelligent systems are also aligned with the features and goals of formative assessment. 
The three main components of intelligent systems - student, expert, and pedagogical 
models - are now briefly described (Shute and Zapata-Rivera, 2007). 

Among them, Smart Tutor is an intelligent instruction system built by researchers at 
Hong Kong University. All behaviors of Smart Tutor are decided by the rule-base which 
consists of expert rules that allow the Planner to determine the selection and ordering of 
materials that should be presented to the learners, and allow the advisor to generate the 
appropriate instructions and tests for the learners (Chen and Hsieh, 2005; Cheung et al., 
2003; Zhang et al., 2001). 

In Depradine (2003), an expert system is used in a so-called code information ex¬ 
tractor (CITOR) to extract syntax and structural information from partially complete or 
incorrect Java code during the implementation phase of the software development cy¬ 
cle. The experimental results demonstrate the effectiveness of the CITOR in specialized 
software development tools such as integrated development environments, intelligent in¬ 
struction systems, and software engineering tools. 

The personalized e-learning system (PELS), based on Item Response Theory, consid¬ 
ers both courseware difficulty and learner ability to provide individual learning paths for 
learners (Chen et al., 2005). This system, however, fails to identify particular learning 
misconceptions for learners. 

Third, we brought attention to the issues of learning diagnosis for web-based instruc¬ 
tion systems (Chen et al., 2008; Hatzilygeroudis et al., 2005; Huang et al., 2007). 

Namba (2012) proposed a diagnosis system for learner’s interest level. His study pre¬ 
sented a neural network model for diagnosis and applied it to 27 students. His study. 
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however, is applicable only to a much too specific domain and is not easy to apply to 
practical situations. 

Based on the embedded concept hierarchy of a test sheet, Cheng et al. (2005) explored 
the possibility of using a hierarchical coding and analytical procedure to the diagnosis of 
individual and class learning and misconceptions. Cheng etal. (2003) focused on creating 
the concept hierarchy by embedding important concepts in a test, and then analyzed the 
results with a hierarchical coding scheme for learning diagnosis. 

In Tsaganou et al. (2003), a fuzzy, case-based reasoning technique is used to construct 
a system for the diagnosis of student cognitive profiles of historical text compression. 
The ability of the system to give similar results when using different historical texts holds 
potential for use in individualized history instruction in an intelligent instruction system. 

Hwang et al. (2003) proposed a computer-assisted approach for teachers to define and 
analyze concept-effect relationships, thus helping them to diagnose students’ learning 
problems. 

Lo et al. (2004) developed an adaptive, hierarchical, concept-level courseware for 
English prepositions. Based on the formative evaluation results from the proposed test 
levels, the system diagnoses learners’ error types in English preposition learning and 
identifies the reasons behind their misconceptions. 

Our work was designed and developed to reflect Instructional Design knowledge, the 
rule-base of Smart Tutor and Lo’s error type diagnosis for advanced studies. The existing 
web-based instruction systems merely give information on whether the learners’ replies 
are ‘correct’ or ‘incorrect’, and only offer evaluations of the learners’ results in terms 
of points. What is needed is a web-based instruction system that diagnoses the learner’s 
comprehension status, and provides cause: why did the learner make the error? Our sys¬ 
tem has a facility that analyses the learner’s weak points and has the ability to diagnose 
the cause of the error, giving advice to the learners and more detailed error information 
than extant systems. 


3. Development of a Web-Based, Intelligent Instruction System 

In this chapter, we will describe the system development environments, system architec¬ 
ture, and system operations with screen shots. 

3.1. System Development Environments 

We developed our system using Windows and the IIS (Internet Information Server) as 
a web server, and MS-SQL for data management. 

We can ask for the HTTP from the client’s side using FORM, Active X, CGI (Com¬ 
mon Gate Interface), or link. 

With the consideration of execution performance in mind, we chose ASP (Active 
Server Page) for web authoring. Until now, the most popular method has been the use of 
CGI; but, when this is executed, new processors arise giving poor performance. ASP is 
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Fig. 1. System implementation environments. 


executed through a DDL (Dynamic Link Library) and does not generate a large function. 
When it is running on the server side, it does not take a reading of the client side. 

The configuration of this system is given in Fig. 1. When the learner requests an 
HTML document in the web browser, the server receives this request through the HTTP. 

All information for evaluation, diagnosis, and advice is stored in the database. The 
system request information server accesses the database using ADO and executes SQL 
queries for data retrieval. The retrieved data is translated into HTML type by an ActiveX 
scripting engine and sent to the client. The learner can then read this information. 

When learners solve the question on the web, the system calls the DB using ASP. 
DBMS retrieves the data through SQL and displays the data on the web in HTTP format. 
Learners’ responses to questions will be stored in the database. 

When the evaluation module is called, the system analyzes the learners’ behavior and 
experts’ knowledge in the database. The system then diagnoses through the VB script 
module and gives advice to the learners in http format. 

3.2. System Architecture Overview 

Our system architecture is given in Fig. 2. Our system comprises of three interface do¬ 
mains and five modules. The three interface domains are subject expert domain, system 
process, and learner interface. In addition, the systems have the following five modules: 

• questions input module, 

• expert knowledge input module, 

• learner answer module, 

• learner diagnosis module, 

• advice module. 

The subject expert domain contains the question input module and expert knowledge 
input module. The system process, interface domain contains the learner diagnosis mod¬ 
ule, while the learner interface domain contains the learner answer module and the advice 
module. 
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Question Table 
Student data Table 


Test Table 
Score Table 
Correct answer Table 
Advice Table 


Fig. 2. System architecture. 


In this system, the learner diagnosis module is the most important module. In the 
learner diagnosis module, we store detailed information about the domain knowledge in 
a database. When the learner makes an error, our system evaluates the error cause using 
domain knowledge in the database. 

3.3. System Operations 

The instructor can insert questions, answers, experts’ knowledge about questions, and 
detailed information about questions; and this information can then be stored in the 
database. Learners then choose the questions and the procedures needed to solve them; 
the system builds up a learner answer database. By running the diagnosis module, the 
system generates a learner evaluation and gives information about that evaluation with 
advice to the learner. 

When the learner solves a question, this system not only analyzes incorrect answers 
but also correct answers in order to constitute the factors involved in solving the ques¬ 
tions. The constituting factors consist of basic concepts, computation, and application; 
this system gives a weight value to the constituent factors used in solving the question. 
The system estimates this weight in a diagnosis module through which the learner’s weak¬ 
nesses can be evaluated. 

3.3.1. Question Input Module 

This module, represented in Fig. 3, is the interface that allows the instructor to input 
questions and data about mathematics into the web. All of this data will be stored in the 
database, and the instructor can manage the data through this interface. The instructor 
can, therefore, make and manage the questions on the web without taking the system into 
consideration. 
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Fig. 3. Question input module. 


98® 52® 97® OlglQ) |7»7(£ 588® 


hH ://loc»t»sVi«la(V»u« «so 

d “S 


s a a 3 

SHi^fas | 




jerr- j— 




fBta: |— 

pan r- 


r* 

Sia |— 

p3b: I 


r* 

a* r~ 




jSf !“■ 

S5b 1 

~jB5c ■ 


j*Mass a* f” 




58 01*98 S* | 

23 'll* "’-J St 1 



aaoi»ass> | 

sa Sx | 



33 98 88 34 | 

S3 S8°»S 34= \ 



01419 * 3 4= | 

*£5 t St | 



989 * 34 | 

I 



P69S | 

QAl“?l | 




Fig. 4. Expert knowledge input module. 


3.3.2. Expert Knowledge Input Module 

This module, found as Fig. 4, stores expert knowledge from expert instructors. This 
knowledge is used for learner diagnosis and advice. Instructors input correct answers 
and await learners’ responses. This expert knowledge will be stored in the database, as 
well. 

The expert knowledge is produced by the expert instructor analysis of subject content. 
Expert knowledge consists mainly of questions and sub-questions. Each question has 
functions and several sub-functions, and each sub-function has weight value. 

As an example, we assume one question such as following: 

There are 5 baskets, and there are 4 eggs in each basket. How many eggs, in total, are 
there in the 5 baskets? 

This question has several sub-questions. 
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SUB Q1: How many eggs are in one basket? 

• When the learner answers correctly, a weight value of two will be given for under¬ 
standing basic concepts of numbers. 

• When the learner answers incorrectly, a weight value of 0 will be given for under¬ 
standing basic concepts of numbers. 

SUB Q2: How many eggs are in two baskets? 

(A) + (B) = (C) 

This question requires an understanding of basic concepts of numbers, an understand¬ 
ing of addition, and knowledge of the computation of addition required to solve the ques¬ 
tion. 

When learners answer these questions, they are classified into seven types and can 
be found in Table 1. Type 2, for example, means that the learner answered correctly for 
A and B, but answered incorrectly for C. For this learner, we give a weight value of 2 
for basic concepts of numbers, 2 for understanding addition and 0 for computation of 
addition. Type 6 and Type 7 mean that the learner answered incorrectly, but Type 6 and 
Type 7 can still be distinguished. Type 7 means the learner provided the wrong answer 
for A, B, and C but answered correctly for computation in addition; a + f3 = 7 . 

SUB Q3; How many eggs, in total, are there in the 5 baskets? 

(A) x (B) = (C). 

When learners answer these questions, they are classified into seven types and can be 
found in Table 1. Type 2, for example, means that the learner answered correctly for A and 
B, but answered incorrectly for C. For this learner, we give a weight value of 2 for basic 
concepts of numbers, 3 for understanding multiplication, and 0 for computation of mul¬ 
tiplication. Type 6 and Type 7 mean that they answered incorrectly, but Type 6 and Type 
7 can still be distinguished. Type 7 means that the learner provided the wrong answer for 
A, B, and C, but answered correctly for computation in multiplication: a x (3 = 7 . 


Table 1 

Learner response types for SUB Q2 


Type 

Domain 






A 

B 

C 

BC 

UA 

CA 

Type 1 

O 

O 

O 

2 

2 

2 

Type 2 

O 

O 

X 

2 

2 

0 

Type 3 

O 

X 

X 

0 

0 

0 

Type 4 

X 

O 

X 

0 

0 

0 

Type 5 

X 

X 

0 

0 

0 

0 

Type 6 

X 

X 

X 

0 

0 

0 

Type 7 

a 

0 

7 

0 

0 

2 


BC: basic concepts of numbers, 
UA: understanding addition, 
CA: computation of addition. 
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Table 2 

Learner response types for SUB Q3 


Type 

Domain 






A 

B 

c 

BC 

UM 

CM 

Typel 

O 

O 

o 

2 

3 

3 

Type2 

0 

O 

X 

2 

3 

0 

Type3 

o 

X 

X 

2 

0 

0 

Type4 

X 

O 

X 

0 

3 

0 

Type5 

X 

X 

o 

0 

0 

0 

Type6 

X 

X 

X 

0 

0 

0 

Type7 

a 

P 

7 

0 

0 

3 


BC: basic concepts of numbers, 
UM: understanding multiplication, 
CM: computation of multiplication. 


When the learner responds to a question, the system accumulates the weight value that 
corresponds to the response type found in Tables 1 and 2. The accumulated weight value 
is used in diagnosis; therefore, based on an analysis of accumulated weight-points in 
sub-functions such as understanding of number sense, understanding of addition, compu¬ 
tation of addition, application of addition, understanding of multiplication, computation 
of multiplication, and application of multiplication, the system provides a diagnosis. 

The extant instruction systems just respond to the correct answer for Type 1, incorrect 
answer for Type 2, Type 3, Type 4, Type 5, Type 6, and Type 7; but our system diagnoses 
in detail through the analysis of the weight-points of the learner responses. This learner, 
for example, is good at computation of multiplication, but lacks ability in the application 
of multiplication. 

3.3.3. Learner Answer Module 

First, the learner inputs his identification and performs learning procedures. The system 
serves questions randomly to the learner through the web browser. The learner responds 
on the web, and this response is stored in the database. Figure 5 is a screen shot of the 
learner answer screen for addition. 

3.3.4. Learner Diagnosis Module 

After the learner finishes solving the questions, the diagnosis module receives informa¬ 
tion on the learner’s answers and expert domain knowledge. The learner’s answers and 
the expert domain knowledge are compared, the learner evaluated, and diagnostic infor¬ 
mation is stored in the database. 

For a learner undertaking a study of ‘multiplication’, the system will evaluate ba¬ 
sic concept understanding, understanding of addition, computation with addition, under¬ 
standing of multiplication, application of multiplication, learning transfer with multipli¬ 
cation, global understanding, global application, and learning transfer. The result of this 
evaluation will be expressed in weight-points. 
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Fig. 5. Learner answer module. 



Fig. 6. Advice module. 


3.3.5. Advice Module 

Through the diagnosis module, a corresponding advice routine can be found. This module 
advises the learner. As can be found in Fig. 6, for example: ’’You have an excellent un¬ 
derstanding of addition, computation of multiplication, and application of multiplication 
for one digit numbers, but you need to study more about applications of multiplication 
for two digit numbers, and the basic concepts of multiplication.” 

4. Conclusion 


We had intended to implement a system that diagnoses learner achievement and gives 
advice. Currently existing web-based instruction systems merely provide learner points, 
but our system provides learners with more detailed information about where the learner 
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is weak and how the learner can improve. This system could apply to other e-learning 
systems and supply useful advice for learners. 

Our system has the following advantages. 

• It takes into account learner understanding status and can provide appropriate 
knowledge for the learner. 

• The proposed system would provide more efficient learning for learners and sup¬ 
port individual learner differences. 

• Each learner would be able to recognize his or her learning status, and would be¬ 
come aware of information that they do not yet know. 

• It compares its learners to other learners on the Internet in order to make compar¬ 
isons with their situations. 

• This research will improve the use of distance education, reduce educational ex¬ 
penses for learners, and will be especially beneficial for disabled and older learn¬ 
ers. 

We need to further develop its mechanisms so that it can be applied to other areas of 
mathematics and, eventually, other areas of learning. We also need to construct a knowl¬ 
edge base for other subject domains of expert knowledge. In the future, we need to do 
further research for systems that supply not only advice but also dynamic instructions for 
learners’ weaknesses. 
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Intelektualios mokymo sistemos, skirtos matematiniams 
skaiciavimams, kurimas 

Du Gyu KIM, Jaemu LEE 

Straipsnyje pristatomas internetu grjstos intelektualios mokymo sistemos, skirtos vidurines 
mokyklos moksleiviams mokytis matematikos, kurimas. Pagrindinis demesys skiriamas intelektu- 
aliems komponentams, galintiems jvertinti esamg padetj ir pateikti patarimus. Egzistuojancios in- 
temetu grjstos mokymo sistemos paprasciausiai pateikia informacijq, ar besimokantieji buvo teisus, 
ar ne, o besimokantieji vertinami atsizvelgiant tik j gautus rezultatus. Tam, kad intemetine sistema 
diagnozuotp besimokanciojo suvokimq, sistema turetp pateikti priezastj, t. y. atsakyti j klausimq, 
kodel besimokantysis padare klaidq. Siuloma sistema turi galimyb? analizuoti besimokanciojo 
silpn^sias vietas ir diagnozuoti klaidos priezastj, patarti besimokanciajam ir pateikti daugiau in- 
formacijos nei egzistuojancios sistemos. Kaupiant informacij^ apie besimokanciojo elgesj ir ana- 
lizuojant besimokanciojo atsakymus, sistema gali pateikti tiksliq, individualizuotq informacij^ ir 
kartu teikti patarimus besimokantiesiems. 



